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Interteach preparation: A comparison of the effects of answering 
versus generating study guide questions on quiz scores 

Helen I. Cannella-Malone 1 , Judah B. Axe 2 , and Edward D. Parker 3 

Abstract: Within an interteaching context, an alternating treatments design was 
used to compare the effects of answering versus writing study guide questions on 
quiz performance in a 10-week methods course in special education. Results 
indicated quiz performance was not substantially influenced by the type of 
preparation, but that writing questions led to slightly higher quiz scores. On a 
social validity questionnaire, most participants reported a preference for 
answering study guide questions because that condition better prepared them for 
quizzes. Interpretations of the results, the socicd validity responses, practical 
implications, and recommendations for future research are discussed. 
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I. Introduction. 

Research indicates attrition rates of approximately 20% for college freshman (Consortium for 
Student Retention Data Exchange, 2003). Examining innovations in college experiences, 
including teaching methods, may reduce this rate. The traditional lecture format of college 
teaching has been criticized for providing limited in-depth learning, having reduced access to 
distance learners, lacking the promotion of practical skills, and facilitating insufficient levels of 
student participation. One alternative to traditional lecture is electronic lectures presented on 
computers (Stephenson, Brown, and Griffin, 2008; Varank, 2006). This has the advantage of 
being accessible to distance learners and systematically programming multiple levels of Bloom’s 
taxonomy (Bloom, 1956; e.g., knowledge, comprehension, synthesis, evaluation). To address the 
criticism of limited provision of practical skills, researchers have evaluated models of 
experiential learning in college classes (Li, Greenberg, and Nicholls, 2007). This technique has 
been shown to promote maintenance of interest and motivation. Co-teaching in college courses 
can provide dual expertise such as, for example, combining a physics professor and a science 
educator in one classroom to prepare physics teachers (Briscoe and Prayaga, 2004). Finally, a 
behavioral approach to increasing student participation in a college lecture is using response 
cards in which students write answers to review questions on dry-erase boards (Kellum, Carr, 
and Dozier, 2001). This form of active student responding increased both participation and quiz 
scores. 

Interteaching (Boyce and Hineline, 2002) is another variation of traditional lecture-based 
college teaching that utilizes principles and practices from the behavior analytic disciplines of 
personalized systems of instruction (Keller, 1968), precision teaching (Lindsley, 1964), and 
cooperative learning (Halpern, 2004). Interteaching is comprised of four components: (a) 
students read assigned course material and answer study guide questions prior to each class 
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session; (b) the instructor focuses a brief lecture on the concepts identified by students as being 
difficult from the previous class session during the first 15 min of class; (c) students then spend 
30 to 40 min grouped in dyads or triads to discuss their answers to the study guide questions; and 
(d) finally, students complete an interteach record, which asks about the quality of the discussion 
and concepts with which they had difficulty. The instructor uses this information to prepare the 
brief lecture for the following class period. 

Previous researchers have found interteaching to result in higher quiz scores and be 
preferred by college students when compared with traditional lecture-based teaching (Saville, 
Zinn, and Elliot, 2005; Saville, Zinn, Neef, Van Norman, and Ferreri, 2006). For example, 
Saville, Zinn, and Elliot compared interteaching to other traditional teaching methods in a 
laboratory setting. In this study, eighty-four undergraduates participated in one of four randomly 
assigned teaching conditions. In the interteach condition, participants were given a study guide 
for an article and 15 min to read the article and answer the study guide questions. They then 
spent 15 min discussing the article with a peer, then 15 min asking the instructor questions about 
the article. In the lecture condition, participants were given a 45 min lecture on the article. In the 
reading condition, participants were given 45 min to read the article. In each of these conditions, 
participants returned after one week to take a 10-question, multiple-choice quiz on the article. In 
the control condition, participants had no exposure to the article and took the quiz on their first, 
and only, visit to the lab. Statistical analyses indicated significant differences between the 
groups, with participants in the interteach group performing significantly better than each of the 
other groups. 

In another study, Saville and his colleagues (2006) conducted two experiments to 
compare interteaching to traditional lecture. Thirty-five graduate students participated in the first 
experiment and 33 undergraduates participated in the second. In both experiments, an alternating 
treatments design was used to examine the effects of interteaching versus traditional lecture on 
short answer quizzes. Results from both experiments indicated that quiz scores following 
interteaching sessions were consistently higher than those following lecture sessions. Results of 
social validity questionnaires suggested that the majority of students preferred interteaching to 
lecture. The authors explained the success of interteaching in terms of active student 
participation in the discussions, social contingencies from peers, opportunities to earn bonus 
points, and guidance of the study guides towards important course material that was linked to 
quiz questions. However, the authors noted that a primary limitation of their analysis was that 
they did not identify which components of interteaching were necessary for improved quiz 
performance (p. 59). 

Although the effects of prepared study guides as one component within the interteaching 
framework have not been examined, the effects of their use in other teaching contexts have been 
evaluated in a number of previous studies. For example, Altus, Welsh, Miller, and Merrill (1993) 
used points and fines as contingencies to increase university housing cooperative students’ 
completion of study guides, which improved their scores on mastery tests. Flora and Fogan 
(1996) found that commercially available, computerized study guides improved the test 
performance of students in general psychology courses as compared with a control group who 
had no study guides. These authors also found that 80% of the experimental participants reported 
enjoying the study guides. Finally, Dickson, Miller, and Devoley (2005) suggested that study 
guides help students focus on critical information. 

Although the available research has found interteaching to be an effective teaching 
methodology, it would be meaningful to analyze the individual components of interteaching to 
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determine the relative effectiveness of each part of the package (Cooper, Heron, and Heward, 
2007). One component that — as an individual independent variable — has been found to have 
positive effects outside of the interteaching context is the study guide (e.g., Altus et al., 1993; 
Dickson et al., 2005; Flora and Logan, 1996). Therefore, the purpose of the current study was to 
extend the research on interteaching by examining the effects of one component of the package, 
the study guide, on quiz scores in an undergraduate special education course. 

II. Method. 

A. Participants, Setting, and Materials. 

Seven college seniors participated in this study. All were Caucasian females, between the ages of 
20 and 22, completing a four-year undergraduate teacher licensure program in special education. 
None of the participants had any previous experience with interteaching, and all students 
enrolled in the course participated in this study. All participants attended each class session of 
the quarter. This study was conducted across eight weeks of a 10-week methods course in special 
education. Classes were held in a classroom on the college campus that was equipped with desks, 
a computer, an LCD projector, a screen, and chalkboards. 

Study guide questions were developed by the instructor for the reading assignments for 
four class sessions and posted on the course website at the beginning of the quarter. Each study 
guide consisted of approximately 10 questions (i.e., two to three questions per reading) based on 
the reading assignments for the week. Students were instructed to respond to the questions and 
bring their typed responses to class. Weekly reading assignments (consisting of three to four 
research articles and book chapters) were posted on the course website or provided in a course 
packet that was available at the beginning of the quarter. Students could access all course 
materials (e.g., readings, study guide questions, assignment guidelines, etc.) at any time during 
the quarter with no restrictions. Interteach records were provided to the students during each 
class session (see Table 1). 

Table 1. Interteach record questions 

Date: 

Duration of Discussion: 

Sufficient time provided: yes no 

On a scale of 1 (not at all) to 5 (very), how prepared was (were) your partner(s)? 

On a scale of 1 (very easy) to 5 (very difficult), how difficult was the material in this unit? 

List topics that gave you difficulty and the nature of the difficulty. 

List issues you would like clarified in the class lecture. 

Provide any additional comments or suggestions. 

B. Dependent Variable. 

The primary dependent measure was the mean percent correct on eight weekly quizzes across 
participants. Quizzes were worth 12 points each and had eight questions based on the weekly 
reading assignments. Each quiz was constructed with the following types of questions: (a) two 
factual multiple-choice questions worth one point each (e.g.. Which of the following strategies is 
a reinforcement-based procedure?), (b) two factual fill-in-the -blank questions worth one point 
each (e.g., The last type of prompt in a least-to-most prompting hierarchy is typically a 
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_.), (c) two factual short answer questions worth two points each (e.g., Describe at 

least two of the strategies presented in this article.), and (d) two interpretive problem solving 
short answer questions worth two points each (e.g.. Describe a strategy you would use to 
increase the social interactions of a child with autism and explain why you would use that 
strategy.). For questions worth one point, no partial credit was given. For questions worth two 
points, students received zero points for incorrect responses, one point for answers that were 
either partially correct or did not answer all parts of the question, and two points if all parts of the 
question were answered correctly. Quiz scores were measured as the mean score out of 12 across 
participants. Mean percent correct on individual question types — multiple choice, fill-in-the- 
blank, factual short answer, and problem solving short answer — were also reported. 

C. Interobserver Agreement. 

Two graders (i.e., two of the authors) independently scored 75% of the quizzes. All three authors 
prepared and reviewed all answers to the quiz questions. They met prior to grading each quiz to 
develop a grading rubric, which was used when grading all quizzes. For example, if the answer 
on the quiz matched the rubric, the answer was scored as correct. For multiple-choice and fill-in- 
the -blank questions, if the answer on the quiz did not match the rubric, it was graded as incorrect. 
For both the factual and problem solving short answer questions, if the answer on the quiz only 
partially matched the rubric, then only one (out of two possible) points was given. Data from the 
two graders were compared for agreements and disagreements. An agreement was scored if the 
two graders assigned the same score to an answer. Any discrepancy between the two graders was 
counted as a disagreement. Interobserver agreement was calculated on a question-by-question 
basis using the formula: (agreements/(agreements + disagreements)) x 100%. Agreement on the 
different types of questions was variable. Interobserver agreement was 100% for the multiple- 
choice questions, 95% (range 78.5-100%) for the fill-in-the -blank questions, 83% (range 64- 
100%) for the factual short answer questions, and 75% (range 57-100%) for the problem solving 
short answer questions. Overall agreement for the quizzes was calculated to be 88% (range 82- 
100 %). 

D. Experimental Design. 

An alternating treatments design was used to examine the effects of the two preparation formats 
on quiz performance (Kazdin, 1982). The study guide assignments were randomly assigned to 
class sessions, and each preparation condition was provided four times over the course of the 
quarter. 

E. Interteaching Procedures. 

Each class session occurred once per week and lasted 2 hr and 18 min and was broken into the 
following schedule: 5 min to address administrative issues, 25 min small group discussions, 5 
min to complete the interteach record, 10 min break, 30 min lecture, 20 min quiz, 30 min 
discussion of practical teaching techniques and discussion of practicum experiences. Similar to 
Saville and his colleagues (2006), participants were randomly assigned to either of two dyads or 
a triad during each class session. They were then given 25 min to discuss either their responses to 
the provided study guide questions or the questions they had prepared. During the first 10 min, 
the participants discussed the material independently. For the remaining 15 min, two graduate 
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teaching assistants circulated through the three groups assisting with any questions and helping 
guide the discussion. Participants were given 25, rather than the traditional 35, min to discuss 
material because this was the amount of time it took them to discuss all of the material for that 
day. After 25 min, their discussions would shift to other, unrelated topics. At the conclusion of 
the discussion, participants were given 5 min to complete an interteach record, which provided 
them with the opportunity to request further clarification on particular topics as well as rate the 
quality of their discussions (see Table 1). Participants were then given a 10 min break, during 
which time the instructor and two graduate teaching assistants reviewed the interteach records to 
determine what material to cover during the short lecture. Following the break, participants were 
given a 30 min lecture on the material, with an emphasis on those topics that they found difficult. 
Unlike a traditional lecture, the 30 min lecture was structured as a group discussion, with 
students asking and responding to questions. Although each class session was 2 hours 18 min, 
only 1 hour 20 min were devoted to the material for that week. 

Answering questions. In the Answer Questions condition, participants were required to 
download a study guide from the course website and provide answers to each question prior to 
class. Each study guide consisted of approximately 10 interpretive questions that required the 
participants to think critically about the material and could not be answered by simply skimming 
the reading (e.g., Why are functional skills for daily living important to teach? Provide 5 
examples of functional skills for daily living and how each benefits a student.). Participants 
answered questions for four randomly assigned class sessions, and were required to bring a hard 
copy of their responses to class to use during discussion. Participants earned points for 
completed study guides, and all participants completed each assignment. 

Writing questions. In the Write Questions condition, participants were required to 
develop their own study guide questions based upon the readings for that week, covering points 
in the reading that they thought were important or presenting issues that they did not understand. 
Participants were instructed to write interpretive questions, rather than questions that could be 
answered by simply skimming the reading, and a model was posted to the course website. 
Participants were required to write at least four questions per reading assignment for four 
randomly assigned class sessions and were required to bring a hard copy of their responses to 
class to use during discussion. Participants earned points for completed study guides, and all 
participants completed each assignment. 

F. Social Validity. 

On the last day of the class, each participant anonymously completed an eight-item questionnaire 
consisting of questions with Likert scale, open ended, and yes/no questions. The questionnaire 
examined the participants’ preferred format of interteach preparation. (See Table 2 for the social 
validity questionnaire.) 
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T able 2. Social Validity Questionnaire. 

1. Do you prefer preparing for interteaching by answering provided study guide questions or 
writing your own questions? Please list reasons for your preference. 


Mostly 

Somewhat 

Preferred Both 

Somewhat 

Mostly Preferred 

Preferred 

Preferred 

Equally 

Preferred 

Answering 

Preparing 

Preparing 


Answering 

Prepared 

Questions 

Questions 


Prepared 

Questions 

Questions 


2. Please rate the extent to which you feel you learned using the interteach method with 
answering provided study guide questions and writing your own questions. Please explain 
why you feel this way. What are some advantages and disadvantages that you experienced 
with each type of preparation (answering prepared questions and writing your own questions)? 


Learned Much 
More With 
Writing 
Questions 


Le aimed Some 
More With 
Writing 
Questions 


Learned Equally 
With Writing & 
Answering 
Questions 


Learned Some 
More With 
Answering 
Prepared 
Questions 


Learned Much 
More with 
Answering 
Prepared 
Questions 


3. How would you rate the extent to which you learned using the interteach method in this course 
as compared to courses that you have taken that did not use the interteach method? 


Learned Much 

Learned Some 

Learned Equally 

Learned Some 

Learned Much 

More With 

More With 

With and 

More Without 

More Without 

interteach 

interteach 

Without 

interteach 

interteach 

interteach 


4. What aspects of the interteach method would you judge as useful? (open ended) 

5. What aspects of the interteach method would you judge as not useful? How would you change 
these aspects? (open ended) 

6. What other feedback do you have about the structure of this class? (open ended) 

7. Would you prefer to participate in interteach in another course? (yes/no) 


8. Would you recommend a class that uses the interteach method to a fellow student? (yes/no) 
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III. Results. 

A. Quizzes. 

The mean percent correct on quizzes across participants is displayed in Figure 1. On quizzes 1, 3, 
6, and 7, which occurred in the Write Questions condition, the class averaged 81% correct (range 
62-93%). On quizzes 2, 4, 5, and 8, which occurred in the Answer Questions condition, the class 
averaged 82% correct (range 73-87%). The last two mean quiz scores in the Write Questions 
condition were slightly higher (i.e., 93% and 90%) than the last two mean quiz scores in the 
Answer Questions condition (i.e., 83% and 87%). 



10 

0 


0 12 3 4 

Quiz 


8 


Figure 1. Overall mean percent correct across eight weekly quizzes. 


Figures 2 through 5 display the mean percent correct on quizzes in the two conditions for 
each question type. On multiple-choice questions (Figure 2), participants averaged 86% correct 
(range 57-100%) in the Write Questions condition and 79% correct (range 57-100%) in the 
Answer Questions condition. On fill-in-the-blank questions (Figure 3), participants averaged 
81% correct (range 79-86%) in the Write Questions condition and 88% correct (range 71-100%) 
in the Answer Questions condition. On factual short answer questions (Figure 4), the average was 
80% correct (range 57-93%) in the Write Questions condition and 88% correct (range 71-100%) 
in the Answer Questions condition. On problem solving short answer questions (Figure 5), the 
average was 81% correct (range 61-93%) in the Write Questions condition and 75% correct 
(range 61-86%) in the Answer Questions condition. Mean quiz scores on problem solving short 
answer questions were consistently higher in the Write Questions condition. 


Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 


28 


Cannella-Malone, H., Axe, J., and Parker, E. 



012345678 

Quiz 


Figure 2. Mean percent correct on the multiple-choice questions across eight weekly 
quizzes. 



012345678 

Quiz 


Figure 3. Mean percent correct on the fill-in-the-blank questions across eight weekly 
quizzes. 
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Figure 4. Mean percent correct on the short answer questions across eight weekly quizzes. 
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Figure 5. Mean percent correct on the problem solving questions across eight weekly 
quizzes. 


B. Social Validity. 


Based on responses to the social validity questionnaire, the majority of the participants preferred 
answering prepared questions, citing improved understanding of the material and feeling that 
these prepared questions would be on the quiz as reasons for this preference. The two 
participants who preferred preparing questions stated that writing questions helped them leam 
more information because it took less time and therefore they could spend more time reading and 
analyzing the articles. They also reported that coming to class with prepared questions was better 
because “everyone wasn’t regurgitating the answers to all the same questions”. The majority of 
the class reported a non-preference for interteaching, stating that they preferred whole class, 
rather than small group, discussions. These responses were similar in nature to the feedback 
provided on the weekly interteach records. While most participants did not like interteaching, 
three out of seven said that they would both take and recommend another course that used 
interteaching. 
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IV. Discussion. 

This research extends the literature on interteaching as a college teaching format (Saville et al., 
2005; 2006). Quiz performance was not substantially influenced by the type of interteach 
preparation — answering study guide questions or writing one’s own questions. The mean percent 
of correct answers to quiz questions across the two conditions differed by only one percent. 
However, visual inspection of the data in the alternating treatments design suggests that the 
Write Questions condition produced slightly higher quiz scores by the end of the course. 
Examining quiz performance across particular question type in the two conditions suggests some 
further findings. Participants scored slightly higher on multiple-choice questions in the Write 
Questions condition and performed substantially higher on fill-in-the-blank questions on two 
quizzes in the Answer Questions condition. On factual short answer questions, there were 
slightly higher scores in the Answer Questions condition and on the problem solving short 
answer questions there were consistently higher scores in the Write Questions condition. 
Differences in quiz scores were examined individually for each student (data not shown) and 
were not different from aggregate differences. 

These results are variable and inconsistent and future research is required to substantiate 
any proposed findings. Nevertheless, potential explanations of the data are offered. One 
interpretation of the data is that the type of interteach preparation — answering study guide 
questions or generating one’s own questions — is a weak variable in the interteaching package 
and does not influence quiz performance in a college course that uses interteaching. On the other 
hand, it may be that having any kind of study guide requirement, either answering provided 
questions or developing questions, could be too closely related to show significant differences. 
Additionally, it may be the case that the interteach discussion or the interteach record are more 
influential variables in the package and that their potency as independent variables should be 
studied. It may also be the case that, especially in a college course with a small number of 
students, the individual components of interteaching are not influential and only the 
contingencies for quiz performance (i.e., one’s grade) influence preparation for class and 
performances on quizzes. Another plausible explanation for the insubstantial difference across 
the conditions is that quiz scores may be an insensitive dependent measure for these independent 
variables. That is, in evaluating preparation type for interteaching, more sensitive dependent 
variables might be the quality of interteach discussion, maintenance of the information over 
months or years, or clinical applications of the concepts from the course. These dependent 
variables are more labor-intensive to measure but might be considered in future studies. 

Despite the small differences across the two conditions, some plausible explanations for 
these differences are offered. The most consistent results were that writing questions produced 
higher scores on short answer problem solving questions, which required participants to 
synthesize and apply the concepts in the readings to educational problems and might require 
greater critical thinking. It is possible that providing students with a study guide with questions 
for students to answer that indicate the important parts of a reading may limit their critical 
thinking. A student’s critical thinking might be enhanced when they are required to generate 
their own questions based on what they consider to be important (Lampert, 2007). Conversely, 
because study guides indicate to students the critical information in text, they might help students 
perform well on factual type questions and this was the case in the fill-in-the-blank and short 
answer questions. However, because writing questions produced slightly higher scores on 
multiple-choice questions, another type of factual question, this interpretation is weakened. 
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A. Limitations. 

A number of limitations should be considered. First, only seven students participated in this 
study. The classes in the two experiments conducted by Saville et al. (2006) had 33 and 35 
students. Because interteaching is conceptualized as an alternative to lecture-based courses, 
which typically have significantly more than seven students, interteaching might be more 
effective with larger classes. Based on the results from the social validity questionnaire, a second 
limitation of the study is that the participants were generally not in favor of the interteaching 
format. This non-preference might have been especially strong with these participants, because 
they had taken two courses with the same instructor in previous quarters in which interteaching 
was not used. Although participants indicated on their interteach records that they did not like the 
interteach arrangement, deviations would have compromised the internal validity of the study. 
The added course requirements over previous course requirements (including in-class 
discussions and more difficult quizzes) also might have influenced the non-preference for 
interteaching and affected the results of the experimental comparison. A third limitation is that 
quiz scores might be an insensitive dependent variable to evaluate the effectiveness of the 
preparation conditions and other possible dependent variables such as quality of discussion and 
clinical applications of the concepts were not measured. A final limitation is that interobserver 
agreement for the problem solving questions was low. This may be due to the fact that the 
students had to discuss how they might use techniques in real-life settings, so there were no 
discrete correct answers. In future studies, it might be beneficial for the different graders to 
further discuss how they might answer the questions in order to develop a more comprehensive 
rubric for this type of question. 

B. Suggestions for Future Research. 

To address these limitations, there are at least four avenues for future research. First, this study 
should be replicated with classes of larger size (e.g., 30 students) at both the undergraduate and 
graduate levels to continue to evaluate the differences between the two preparation formats. 
Second, future studies should compare the use of study guides with no study guides and no other 
contingencies for preparation. This would provide a clearer evaluation of the effects of the study 
guide. Third, other dependent variables should be measured. These might include a measure of 
the quality of class discussion, quizzes with a higher percentage of problem solving and 
application questions, the clinical application of concepts, the designing of educational goals 
based on the concepts, and orally explaining concepts and applications. A fourth area for future 
research is to evaluate the effects of the other components of interteaching such as the in-class 
discussion or the interteach records. Data on the effects of these individual components may 
provide a more complete account of the construct of interteaching. 

C. Implications for Practice. 

Despite the methodological limitations of this study and the need for further research, some 
implications for practice are suggested. Based on this and previous studies, it appears that 
interteaching is a reasonable alternative to lecture -based formats of college teaching that 
provides many benefits to both students and instructors as outlined by Boyce and Hineline 
(2002) and Saville et al. (2006). Although there were not substantial differential effects of the 
two preparation strategies evaluated in this study, one conclusion is that contingencies for any 
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type of class preparation, including reading text and writing about it, help students learn content. 
Further, participants may perform better on quizzes that require critical thinking if they critically 
examine course reading and identify areas of importance. Finally, until there is more component 
analysis data, interteaching should be implemented as described by Boyce and Hineline (2002) 
and Saville and colleagues (2005; 2006). 

D. Conclusions. 

In conclusion, answering study guide questions or writing one’s own questions based on course 
reading did not substantially differentially affect quiz scores in a college course using 
interteaching. It is possible that generating one’s own questions produces higher scores on 
problem solving questions that require critical thinking, but future research is needed to support 
that conclusion, interteaching can be an effective, behavior analytic alternative to lecture-based 
teaching and should continue to be researched. 

Appendix 

Appendix 1. Sample Study Guide Questions Provided by the Instructor. 

1. What are the differences between discrete trial training and embedded instruction? Describe 
at least three advantages and disadvantages of each. 

2. Discuss the importance of examining the effectiveness of an intervention that is implemented 
by the classroom teacher and his/her paraprofessionals? 

3. How was errorless learning used in this study? Would you use this strategy in your 
classroom? Why or why not? 

Appendix 2. Sample Study Guide Questions Written by the Participants. 

1. How was the functional analysis conducted and what were the results? The article does not 
go into very much detail of this aspect of the experiment. It is not as important as what was 
done or the results, but it would have been nice to mention. 

2. What is an analogue session? I have never heard of this term before. 

3. How/why was each of the six assessments chosen? What was each trying to show? 

4. Describe the graph, which depicts the results and what it means for the experiment? 
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